Body movement and respiration monitor

ABSTRACT

A monitor which can detect respiration of a sleeping person without being affected by the attitude of the sleeping person or the indoor illumination light and can easily evaluate detected respiration quantitatively through image measurement. The monitor comprises means ( 1 ) for projecting a specified illumination pattern, means ( 5 ) for picking up the image of projected light continuously, means ( 7 ) for calculating inter-frame moving amount of the illumination pattern from the image of two frames acquired by the image pickup means at different times, means ( 8 ) for generating a moving amount waveform data comprising inter-frame moving amounts arranged in time series, and means ( 9 ) for detecting the movement of an object from the moving amount waveform data.

The Priority document PCT/JP01/10685 filed on Dec. 6, 2001 is hereinincorporated in its entirety into this specification.

TECHNICAL FIELD

The invention of this application relates to a monitor. Moreparticularly, the present invention relates to a monitor capable ofmonitoring the body movement etc., of a sleeping person in a noncontactmanner with image measurement.

BACKGROUND ART

With the advent of an aging welfare society, the importance of homehealth monitoring for disease prevention and self care is expected tocontinue to grow, and various bioinstrumentation methods have beenproposed. An aged but healthy normal person who can live an ordinarylife can be mentioned as an example of a person to be the subject ofsuch home health monitoring. However, under the present circumstances,such a person lacks the motivation to exercise health control andperform a troublesome device operation, because he/she does not needimminent health care. Therefore, a proposal has been made of variousbioinstrumentation devices capable of measuring biomedical informationunder an unconscious condition without any device operation. Asmeasurement methods employed in such bioinstrumentation devices, thereare known a method of arranging temperature sensors in a bed or a bottommattress and, from a temperature distribution, recording body movementsat the time of getting into bed, at the time of rising from bed, andduring sleep in bed, a method of deriving an electrocardiogram from anelectrically conductive sheet and a pillow cover, and a method ofmeasuring respiration and heart beat during sleep by use of a loadsensor such as a highly accurate strain gauge or a load cell.

Recently, a proposal has been made to use such bioinstrumentationtechnology as a security device applied in the safety confirmation of anaged person besides the usage of health monitoring. For example, aleaving-bed sensor has been generally sold for detecting the leaving orfalling out of bed of a person during sleep by measuring anelectrostatic capacity etc. Additionally, an apnea detector is wellknown in which an apnea state, caused by an apnea syndrome by whichbreathing ceases during sleep, is detected by measuring respiration witha pressure-sensitive sensor, and a third party can be informed of thestate (e.g., Japanese publication of unexamined patent application No.(hereinafter referred to as JP-A-) 2000-107154).

As a matter of course, respiration provides a very useful clue forimmediately knowing the health condition of a person. Respirationmeasurement performed during sleep is expected to be useful not only indetecting an apnea syndrome but also in discovering a spasmodic incidentcaused by a sudden attack during sleep. As mentioned above, a typicalrespiration detecting method during sleep is a method of detectingrespiration based on time base measured values of a lead sensor or apressure-sensitive sensor. Additionally, a method of using a vibrationsensor, a radio-wave sensor, or air-pressure detection is publicly known(e.g., JP-A-H7-327939, JP-A-H11-28195, and JP-A-2000-83927). In thesemethods, since a measured signal is weak, a high-performance signalamplifier or some kind of signal processing is required to acquire anddetect a stable signal, and, as a system, it becomes expensive and largein scale.

On the other hand, some proposals have been also made of a method ofacquiring an image of a sleeping person by use of an image pickup deviceand detecting respiration based on the acquired image. With recentdevelopments in electronic equipment, a high-performance image pickupdevice has appeared on the market at an extremely low price, and, sincethe device has noncontact properties, the method of detectingrespiration based on an image has been brought to public attention as atechnique having high practical usefulness.

For example, in “Image-processing device and patient-observing device”of JP-A-H11-86002 and “Region-of-interest setting device of respirationmonitoring and respiration-monitoring system” of JP-A-H11-225997, thebasic features of those inventions is to monitor respiration byexamining a difference between images acquired in different time by theimage pickup device.

The “image-processing device and patient-observing device” ofJP-A-H11-86002 is composed of a TV camera, a respiration monitoringdevice, and a local-region automatic setting processing device. Thelocal-region automatic setting processing device is composed of an edgedetecting section that detects an edge included in a local region imageset on an image for which a patient who requires care is photographed bythe TV camera, a brightness distribution measuring section that measuresthe brightness distribution of each local partial region image dividedby the edge in the local region image, and a determining section thatsets a local region image to extract movement information by analyzinginformation about a detected edge and information about a measuredbrightness distribution. The local region image is divided into aplurality of local partial regions that are identical in brightness, anda time differentiation process is applied to each pixel included in eachlocal partial region, and the total amount of the time differentiationis calculated. A time base change of this amount is analyzed, and aperiodic appearance is detected as respiration, whereas irregularitiesin the period and amplitude are detected as great body movements, suchas a body twist.

In the “Region-of-interest setting device of respiration monitoring andrespiration-monitoring system” of JP-A-H11-225997, a calculation isfirst performed of the absolute value of a difference between every oneframe of a plurality of frame images picked up by a CCD camera over ½periods of respiration. Thereafter, the difference images are integratedand stored, the positions and sizes of variation regions are thencalculated from variation information that has been integrated andstored, and they are set as temporary regions in order from the largestto the smallest region in the variation ones. Thereafter, a judgment ismade of whether a concentration-value histogram, that shows thedistribution of the number of pixels of each concentration value,exhibits a two-peak characteristic having a height greater than apredetermined value in the temporary regions and whether the area valueof the variation regions is greater than a given value. If thiscondition is satisfied, the temporary region is set as a region ofinterest (abbreviated as ROI). Further, a time differentiation processis performed in the set ROI, the absolute value of the differencebetween each pixel is then obtained, and a surface integral is applied.The surface integral is performed in a time series manner, and, like theinvention of JP-A-H11-86002, a time base change in this surface integralis analyzed, and a periodic appearance is detected as respiration,whereas irregularities in the period and amplitude are detected as greatbody movements, such as a body twist.

Further, a method of detecting respiration by an optical flow ofmovement of a sleeping person is known as an image-using technique otherthan the aforementioned methods. The optical flow is characterized bydetecting the movement of a sleeping person as a velocity vector, and arespiration waveform having a periodic rhythm and a body-movementwaveform having a high peak can be obtained from a vector field byemploying the fact that most upward vectors are detected in inspirationwhereas most downward vectors are detected in expiration.

The respective methods described above are to observe the movement ofshadows on a quilt by use of illumination light, and there remains thefundamental problem of being sometimes incapable of detecting themovement of shadows depending on lighting conditions, the posture of asleeping person, or the design of the quilt. Additionally, since animage pickup device must be set close to the sleeping person in order tophotograph the shadow on the quilt, the respiration monitoring accordingto the aforementioned methods is considered to entail a psychologicallyoverpowered feeling when the person to be monitored goes to bed.

Additionally, it is said that the method based on a time differentiationcan evaluate a frequency of the movement of a targeted person, butcannot make a quantitative evaluation of the movement thereof. Incontrast, in the optical-flow method, the optical flow enables aquantitative evaluation of the movement of a sleeping person, but, inpractice, much computation time is needed to calculate the optical flow,and there remains the problem of requiring expensive processingequipment.

DISCLOSURE OF INVENTION

Therefore, the invention of this application has been made inconsideration of the foregoing circumstances, and it is an object of theinvention to provide a monitor capable of detecting respiratory movementof a sleeping person without being affected by the posture of thesleeping person or the indoor illumination light and capable of easilyevaluating detected respiratory movement quantitatively through imagemeasurement.

The monitor of the present invention comprises lighting patternprojecting means for projecting a specified lighting pattern, imagepickup means for picking up light of projected wavelength continuously,moving amount calculating means for calculating an inter-frame moving amount of the lighting pattern from two frames of images acquired by theimage pickup means at different times, moving amount waveform generatingmeans for generating moving amount waveform data comprising inter-framemoving amounts arranged in time series, and waveform detecting means fordetecting a movement of an object to be picked up from the moving amountwaveform data.

The waveform detecting means is to detect the body movement andrespiratory movement of a sleeping person, and the monitor furthercomprises safety deciding means for deciding the safety of the sleepingperson from the body movement and respiratory movement of the sleepingperson and signal outputting means for, when the safety deciding meansdecides that the sleeping person is in a dangerous state, outputting asignal indicating it.

The moving amount calculating means can easily calculate the movingamount of the lighting pattern by calculating the inter-frame movingamount of the lighting pattern in an axial direction connecting thelighting pattern projecting means and the image pickup means.

The waveform detecting means can detect a respiratory pattern and a bodymovement pattern by detecting a periodic pattern as the respiratorypattern and by detecting a variation having a high peak as the bodymovement pattern from moving-amount waveform data.

The safety deciding means can decide a dangerous state of a sleepingperson by deciding that the sleeping person is in a dangerous state whena period of the respiratory pattern falls into disarray in a short time,when the period of the respiratory pattern suddenly changes, when therespiratory pattern continuously disappears, or when the body movementpattern frequently appears in a short time.

The moving amount calculating means can easily and reliably detect themovement of an object to be photographed by calculating an inter-framemoving amount of the lighting pattern in an axial direction connectingthe lighting pattern projecting means and the image pickup means whileproviding a positive or negative sign in accordance with a movingdirection.

The waveform detecting means can reliably detect a respiratory patternby, from among respiratory patterns, detecting a value having a positiveor negative sign corresponding to a movement in a direction from thelighting pattern projecting means to the image pickup means on an imageand a value having a positive or negative sign corresponding to amovement in a direction from the image pickup means to the lightingpattern projecting means on the image as an expiratory pattern and as aninspiratory pattern, respectively, or vice versa.

The waveform detecting means can well adapt to processing in a computerby counting zero crosses where signs are reversed between an expiratorypattern and an inspiratory pattern to count respirations.

The waveform detecting means can further exactly calculate a respirationnumber per unit time or a respiration period by calculating therespiration number per unit time or the respiration period from thecounted respirations.

The waveform detecting means can further easily and exactly calculate arespiration number per unit time or a respiration period by conducting afrequency analysis of moving-amount waveform data or sampling dataobtained by sampling respiratory patterns, by calculating the mostprominent frequency from a resulting frequency spectrum, and bycalculating the respiration number per unit time or the respirationperiod from this frequency.

The waveform detecting means can further calculate the respirationnumber per unit time or the respiration period through digital signalprocessing by conducting a frequency analysis with discrete Fouriertransform or discrete wavelet transform.

The safety deciding means can reliably decide that a sleeping person isin a dangerous state by deciding whether respiration belongs to normalrespiration, Cheyne-Stokes respiration, central hyperventilation, ataxicrespiration, or Kussmaul respiration from expiratory patterns andinspiratory patterns and by deciding that the sleeping person is in adangerous state when the respiration belongs to any one of Cheyne-Stokesrespiration, central hyperventilation, ataxic respiration, and Kussmaulrespiration.

The moving amount calculating means calculates an inter-frame movingamount in a partial region of the lighting pattern in an axial directionconnecting the lighting pattern projecting means and the image pickupmeans, and determines a region in an image where an inter-frame movingamount in the partial region appears much greater than a predeterminedamount or where an inter-frame movement in the partial region appearsfrequently beyond a predetermined amount as a region where the sleepingperson exists, and the safety deciding means can safely and reliablydecide that the sleeping person is in a dangerous state by deciding thatthe sleeping person is in a dangerous state when the region where thesleeping person exists moves in a shorter time than a predetermined timeand moves frequently beyond a predetermined amount.

The moving amount calculating means calculates an inter-frame movingamount in a partial region of the lighting pattern in an axial directionconnecting the lighting pattern projecting means and the image pickupmeans, and determines a region in an image where an inter-frame movingamount in the partial region appears much greater than a predeterminedamount or where an inter-frame movement in the partial region appearsmore frequently than a predetermined frequency as a region where thesleeping person exists, and the safety deciding means can reliablydecide that the sleeping person is in a state of imminently falling froma bed by deciding that the sleeping person is in a dangerous state whenthe region where the sleeping person exists is brought closer to one endof the bed than a predetermined distance.

Additionally, it is possible to know that the sleeping person is in adangerous state at a location remote therefrom by providing informingmeans for, based on a signal output from signal outputting means,informing a third party that the sleeping person is in a dangerousstate.

The informing means can urge a third party to swiftly deal with thesituation by informing the third party that the sleeping person is in adangerous state by voice, characters, symbols, intensity of lightincluding interior illumination light, or vibrations through atelecommunication line.

The lighting pattern projecting means projects a lighting pattern thathas a plurality of lighting spots, and the moving amount calculatingmeans calculates an inter-frame moving amount of each lighting spot inan axial direction connecting the lighting pattern projecting means andthe image pickup means while providing a positive or negative sign inaccordance with a moving direction, and calculates an inter-frame movingamount of the lighting pattern by use of a moving amount of one or moreof the lighting spots, whereby the moving amount of the lighting patterncan be easily calculated.

The lighting pattern projecting means projects a lighting pattern thathas a plurality of lighting spots, and the moving amount calculatingmeans calculates an inter-frame moving amount of each lighting spot inan axial direction connecting the lighting pattern projecting means andthe image pickup means while providing a positive or negative sign inaccordance with a moving direction, and calculates the total movingamount of each lighting spot as an inter-frame moving amount of thelighting pattern, whereby a statistical noise influence can be reduced.

The lighting pattern projecting means projects a lighting pattern thathas a single slit ray or a plurality of slit rays, and the moving amountcalculating means calculates an inter-frame moving amount of eachlighting spot in an axial direction connecting the lighting patternprojecting means and the image pickup means while providing a positiveor negative sign in accordance with a moving direction, and calculatesan inter-frame moving amount of the lighting pattern by use of a movingamount of a pixel corresponding to the single or plural slit rays,whereby the moving amount of the lighting pattern can be easilycalculated.

The lighting pattern projecting means projects a lighting pattern thathas a single slit ray or a plurality of slit rays, and the moving amountcalculating means calculates an inter-frame moving amount of each pixelcorresponding to a slit ray in an axial direction connecting thelighting pattern projecting means and the image pickup means whileproviding a positive or negative sign in accordance with a movingdirection, and calculates the total moving amount of each pixelcorresponding to the slit ray as an inter-frame moving amount of thelighting pattern, whereby the number of measurement points can beincreased, and a statistical noise influence can be reduced.

The lighting pattern projecting means is disposed directly above a partclose to one end of an edge of bedding facing a head of or a foot of aperson sleeping on the bedding, and the image pickup means is disposeddirectly above a part close to an opposite end thereof, wherebymonitoring can be performed without allowing the sleeping person to havean intense consciousness of being monitored.

The provision of presence-in-bed detecting means for detecting thepresence/absence of a sleeping person makes it possible to swiftly andaccurately judge whether the sleeping person is absent or the sleepingperson has stopped his/her breathing.

The provision of a pressure sensitive switch that can be placed underthe sleeping person and can be used to detect the presence/absence ofthe sleeping person makes it possible to swiftly and accurately judgewhether the sleeping person is absent or the sleeping person has stoppedhis/her breathing.

This specification includes the contents of the description and/or thedrawings of Japanese Patent Application No. 2000-372820 on which thepriority of this application is based.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic diagram showing the structure of a monitoraccording to the invention of this application;

FIG. 2 is a schematic diagram explaining the calculation of aninter-frame moving amount of a lighting pattern on an image in a movingamount calculating means;

FIG. 3 is a schematic diagram showing the principle of the monitoraccording to the invention of this application;

FIG. 4 shows an example of a waveform pattern of moving-amount waveformdata;

FIG. 5 is a schematic diagram showing the principle of the respiratorymovement detection of the monitor according to the invention of thisapplication;

FIG. 6( a), FIG. 6( b), FIG. 6( c), and FIG. 6( d) show examples ofnormal and abnormal respiratory patterns;

FIG. 7 is a schematic diagram showing a lighting pattern projected by alighting pattern projecting means;

FIG. 8 is a schematic diagram showing a lighting pattern projected bythe lighting pattern projecting means;

FIG. 9 is a schematic diagram showing an arrangement of the lightingpattern projecting means and an image pickup means;

FIG. 10 is a schematic diagram showing an arrangement of the lightingpattern projecting means and the image pickup means;

FIG. 11 is a schematic diagram showing an arrangement of the lightingpattern projecting means and the image pickup means;

FIG. 12 is a schematic diagram showing the structure of an indoor safetymonitor that is the invention of this application; and

FIG. 13 is a schematic diagram showing the structure of an indoor safetymonitor that is the invention of this application.

BEST MODE FOR CARRYING OUT THE INVENTION

The invention of this application has the aforementioned features, andembodiments thereof will hereinafter be described.

FIG. 1 is a schematic diagram showing the structure of a monitoraccording to the invention of this application. The monitor 6 iscomposed of a lighting pattern projecting means 1, an image pickup means5, a moving amount calculating means 7, a moving amount waveformgenerating means 8, a waveform detecting means 9, a safety decidingmeans 10, a signal outputting means 11, and an informing means 12. Alighting pattern 4 is first projected from the lighting patternprojecting means 1 onto a body 2 of a sleeping person or onto bedding 3.It is preferable to set the wavelength of projected light to be infraredrays, so as to project without allowing the sleeping person to haveconsciousness of being monitored. The lighting pattern 4 projected tothe body 2 or the bedding 3 is continuously picked up as an image by theimage pickup means 5. The image pickup means 5 can pick up, for example,an infrared ray that is a wavelength of the projected light. In responseto a movement in an optical axial direction of the image pickup meansresulting from a movement of the body 2 or resulting from a movement ofthe bedding 3 responding to the movement of the body 2, a movement of alighting pattern, whose optical axis is different from that of the imagepickup means, in an pickup image plane occurs, and an inter-frame movingamount of this lighting pattern is calculated by the moving amountcalculating means 7 from two frames of images different in time thathave been acquired by the image pickup means 5.

Thereafter, in the moving amount waveform generating means 8, movingamount waveform data is generated by arranging the inter-frame movingamounts calculated by the moving amount calculating means 7 in timeseries. Further, in the waveform detecting means 9, the body movementand respiratory movement of the sleeping person are detected from themoving amount waveform data. Even 2 or 3 frames/second of images to beacquired are enough to generate the waveform of respiration. Preferably,the images are acquired at regular intervals in order to generate themoving amount waveform data.

The monitor 6 of the invention of this application may be provided withthe safety deciding means 10 by which the safety of a sleeping person isdecided from the body movement and respiratory movement of the sleepingperson detected by the waveform detecting means 9. When it is decidedthat the sleeping person is in a dangerous state in the safety decidingmeans 10, the signal outputting means 11 outputs a signal indicating it.As a result, the dangerous state can be recorded, for example, togetherwith its time. Further, based on the signal output by the signaloutputting means 11, the informing means 12 automatically informs athird party the sleeping person is in a dangerous state.

FIG. 2 is a schematic diagram explaining the calculation of aninter-frame moving amount of a lighting pattern on an image in themoving amount calculating means. In the moving amount calculating means7, an inter-frame moving amount of the lighting pattern in an axialdirection connecting the lighting pattern projecting means 1 and theimage pickup means 5 is calculated from two frames different in timeamong images acquired by the image pickup means 5.

FIG. 3 is a schematic diagram showing the principle of the monitoraccording to the invention of this application. In an opticalarrangement shown in FIG. 3, the following equation is established.δ=dlZ/{h(h−Z)}where

δ: inter-frame moving amount of a lighting pattern,

h: installation heights of the lighting pattern projecting means and theimage pickup means,

d: distance between the lighting pattern projecting means and the imagepickup means,

l: focal length of the image pickup means, and

Z: displacement amount in the height direction.

That is, the inter-frame moving amount δ of the lighting patterncorresponds to a displacement amount Z in the height direction resultingfrom a movement of the body 2 of a sleeping person or resulting from amovement of bedding 3 in response to the movement of the body 2.Therefore, in the waveform detecting means 9, a waveform showing aperiodic pattern can be detected as a respiratory pattern, and awaveform showing a variation having a high peak can be detected as abody movement pattern (i.e., pattern in which the waveform is varied bya body movement, such as turning over in bed) separately from each otherin the moving amount waveform data generated by the moving amountwaveform generating means 8. In the respiratory pattern, a respirationnumber per unit time can be known from its periodicity. Further, thetime transition of the periodicity shows the stability of therespiration of the sleeping person. Further, since the inter-framemoving amount of the lighting pattern corresponds to a displacementamount in the height direction as mentioned above, it serves also as ameans for knowing the depth of the respiration.

A predominant frequency component can be calculated by conducting afrequency analysis of moving amount waveform data or respiratory patternwith discrete Fourier transform or discrete wavelet transform, and arespiration number per unit time can also be calculated from the valueof this frequency. Fast Fourier transform can be mentioned as a typicalexample of discrete Fourier transform. For example, on the assumptionthat the most prominent sample in a result obtained by applying fastFourier transform to the respiratory pattern is the M-th sample insampling data in which N samples of respiratory patterns have beensampled at sampling intervals of Δt second, the most prominent frequencyf can be obtained by M/(Δt×N), and a respiration number per minute canbe obtained by 60×f. A respiration period can be obtained by 1÷f. As amatter of course, various frequency analysis techniques other than fastFourier transform can be applied in the invention of this application.

In the safety deciding means 10, for example,

(i) when the period of a respiratory pattern falls into disarray in ashort time,

(ii) when the period of a respiratory pattern suddenly changes,

(iii) when a respiratory pattern continuously disappears, and

(iv) when a body movement pattern frequently appears in a short time,

it is decided that a sleeping person is in a dangerous state. Thecondition of (i) or (ii) is considered to be caused by an infirmity ofthe lungs, such as spontaneous pneumothorax or bronchial asthma, acardiopathy, such as congestive heart failure, or a cerebrovasculardisease, such as cerebral hemorrhage. The condition of (iii) appearswhen respiration stops. A possible condition of (iv) is the fact thatthe sleeping person suffers from some reason and is in distress.

In the moving amount calculating means 7, preferably, an inter-framemoving amount of a lighting pattern in an axial direction connecting thelighting pattern projecting means 1 and the image pickup means 5 iscalculated from two frames of images different in time while applying apositive or negative sign in accordance with a moving direction. Hereby,concerning the respiration, an expiratory pattern and an inspiratorypattern shown in FIG. 4 are always obtained, and the respiration can becounted by measuring the number of “zero crosses” (intersections wherethe sign is reversed) appearing from the expiratory pattern to theinspiratory pattern or from the inspiratory pattern to the expiratorypattern. This is advantageous when processing is performed by acomputer.

FIG. 4 shows an example of a waveform pattern of moving-amount waveformdata. The moving amount waveform data generated by the moving amountwaveform generating means 8 shows a waveform pattern, for example, shownin FIG. 4.

FIG. 5 is a schematic diagram showing the principle of the respiratorymovement detection of the monitor according to the invention of thisapplication. In an optical arrangement shown in FIG. 5, when the heightchanges upward “A,” a lighting pattern on an image moves in a direction“A″” from the image pickup means to the lighting pattern projectingmeans. In contrast, when the height changes downward “B,” the lightingpattern on the image moves in a direction “B″” from the lighting patternprojecting means to the image pickup means on the image.

Therefore, concerning the inter-frame moving amount of the lightingpattern, if moving a mount waveform data is generated from theinter-frame moving amount calculated while having a positive or negativesign in accordance with the moving direction, and if a waveform showinga periodic pattern among the moving amount waveform data is detected asa respiratory pattern, a part 41 having a sign corresponding to amovement in the direction “A′” can be detected as an expiratory pattern,whereas a part 42 having a sign corresponding to a movement in thedirection “B′” can be detected as an inspiratory pattern among thedetected respiratory patterns.

Since the moving direction of a lighting pattern follows a horizontal orvertical direction in the picked-up image by setting the horizontal orvertical direction of a picked-up image to coincide with the axialdirection connecting the lighting pattern projecting means and the imagepickup means, the moving amount of the lighting pattern can be easilycalculated.

Further, a respiration number per unit time (or respiration period) canbe known by a time point where the part 41 having a sign correspondingto a movement in the direction “A′” and the part 42 having a signcorresponding to a movement in the direction “B′” each appear once as asingle breath.

In the safety deciding means, a decision may be made as to whether therespiration of a sleeping person is a normal respiration by pre-storingeach respiratory pattern concerning normal and abnormal respirations andcomparing these with a respiratory pattern of the sleeping person.

FIG. 6( a), FIG. 6( b), FIG. 6( c), and FIG. 6( d) show examples ofnormal and abnormal respiratory patterns. A normal respiratory patternto be registered is a periodic pattern shown in FIG. 6( a). On the otherhand, a respiratory pattern, such as Cheyne-Stokes respiration, centralhyperventilation, ataxic respiration, or Kussmaul respiration, that isconsidered to occur when a physiologically disorder arises in the body,is registered as an abnormal respiratory pattern. As an example, arespiratory pattern of Cheyne-Stokes respiration is shown in FIG. 6( b),a respiratory pattern of central hyperventilation is shown in FIG. 6(c), and a respiratory pattern of ataxic respiration is shown in FIG. 6(d). These respiratory patterns are obviously different in the waveformfrom the normal respiratory pattern, and a decision is made as to arespiratory pattern to which that of a sleeping person belongs on thebasis of the fact that they are different in the frequency ofrespiration, the occurrence count thereof, and the depth thereof.

When it is decided that the respiration of the sleeping person belongsto a respiratory pattern that is considered to occur whenphysiologically disorders arise in the body, the safety deciding meansdecides that the sleeping person is carrying out abnormal respirationand is in a dangerous state. When the sleeping person displays abnormalrespiration, the informing means may inform a third party about the nameof the respiratory pattern of the sleeping person, the name of a diseaseconsidered to cause the respiration, the disease organ, the diseasepart, etc. Table 1 shows the disease name or disease part when theabnormal respiratory pattern occurs.

TABLE 1 Cheyne-Stokes respiration Disorder under both-sides cerebralcortex and of diencephalon Central hyperventilation Disorder from lowermidbrain to upper pons Ataxic respiration Disorder from lower pons toupper medulla oblongata Kussmaul respiration Diabetic coma or uremia

FIG. 7 and FIG. 8 are each a schematic diagram showing a lightingpattern projected by the lighting pattern projecting means. Preferably,a lighting pattern projected by the lighting pattern projecting meanshas a spatially discrete distribution. For example, a plurality oflighting spots 71 shown in FIG. 7, a single slit ray 81 shown in FIG. 8,or a plurality of slit rays 82 are selected and used.

In a lighting pattern having a plurality of lighting spots, aninter-frame moving amount of each lighting spot is calculated whileadding a positive or negative sign in accordance with a movingdirection, and the total moving amount of each lighting spot iscalculated as an inter-frame moving amount of the lighting pattern.Likewise, in a lighting pattern having a single or a plurality of slitrays, an inter-frame moving amount of each pixel corresponding to theslit ray is calculated while adding a positive or negative sign inaccordance with a moving direction, and the total moving amount of eachpixel corresponding to the slit ray is calculated as an inter-framemoving amount of the lighting pattern. A statistical noise influence canbe reduced by calculating the total inter-frame moving amount whenspatially discrete lighting patterns are used.

In the moving amount calculating means, a region in an image where theinter-frame moving amount of a lighting pattern appears greatly or wherean inter-frame movement of the lighting pattern appears frequently isdetermined as a region where the sleeping person exists. When thisregion frequently moves in a short time, the safety deciding means maydecide that the sleeping person is in a dangerous state. This can beregarded as a condition where the sleeping person suffers from somereason and is in distress.

The safety deciding means may decide that the sleeping person is in adangerous state when the region where the sleeping person exists isbrought extremely close to one of the sides of a bed and when thesleeping person on the bed is being monitored. This can be regarded as asituation where the sleeping person is in a dangerous position as if tofall from the bed.

In the monitor of the invention of this application, preferably, a rangewhere the lighting pattern is projected is set within a range coveringpositions that can be occupied by the belly, chest, back, and shouldersof the sleeping person. Likewise, preferably, a range of a regionphotographed by an image pickup device is set within a range coveringpositions that can be occupied by the belly, chest, back, and shouldersof the sleeping person.

FIG. 9 and FIG. 10 are each a schematic diagram showing an arrangementof the lighting pattern projecting means and the image pickup means. Forexample, as shown in FIG. 9, the lighting pattern projecting means 91 isdisposed directly above a part close to the head 92 or the foot 93 ofthe sleeping person, and the image pickup means 94 is disposed directlyabove a part around the belly 95 of the sleeping person. At an edge 104of bedding 101 placed under the sleeping person facing the head 92 orthe foot 93 of the sleeping person as shown in FIG. 10, the lightingpattern projecting means 106 may be disposed directly above a part closeto one end 105 of the edge 104, and the image pickup means 108 may bedisposed directly above a part close to an opposite end 107 of the edge104. The arrangement shown in FIG. 9 can capture the respiratorymovement of the sleeping person extremely sensitively, and has theadvantage of being able to detect a detailed respiratory pattern.However, since the image pickup device comes into view of the sleepingperson, there can be a case in which the arrangement is accompanied by apsychologically oppressive feeling. In contrast, an arrangement shown inFIG. 10 cannot specifically catch the respiration unlike the arrangementshown in FIG. 9, but this arrangement is considered to have a weakerpsychologically oppressive feeling. Of course, the arrangement shown inFIG. 9 can be formed to sacrifice slightly the sensitivity ofrespiratory movement as in FIG. 11, and an oppressive feeling may bereduced by disposing the lighting pattern projecting means 111 and theimage pickup means 112 at one side end 113 of the bedding. Thesearrangements are to be appropriately selected depending on thesituation.

Any type of lighting pattern projecting means can be used if thelighting pattern projecting means can project light spatiallydiscretely. Use can be made of, for example, a fiber grating, adiffraction grating, a lens array, a formed image of a light sourcearray or an aperture array, a device that collimates an outgoing beam ofthe light source array or the aperture array, etc.

In the monitor of the invention of this application, as an auxiliarymeans for accurately detecting whether a sleeping person is present orabsent, a pressure sensitive switch may be disposed in the beddingplaced under the sleeping person. The presence/absence of the sleepingperson is determined by ON/OFF of this switch.

Further, in the monitor of the invention of this application, theinforming means has a voice outputting function and informs a thirdparty by voice that the sleeping person is in a dangerous state. It mayinform the third party by characters, symbols, intensity of lightincluding interior illumination light, or vibrations. Further, theinforming means may have a function to connect to a telecommunicationsline, such as a general telephone line, ISDN line, PHS line, or cellulartelephone line, and may inform the third party by voices, characters, orsymbols that the sleeping person is in a dangerous state.

FIG. 12 is a schematic diagram showing an arrangement of the lightingpattern projecting means and the image pickup means. The principle ofthe invention of this application is not applied limitedly to a monitorfor a sleeping person, of course. For example, as a matter of course,the principle can be applied also to an indoor safety monitor thatmonitors the safety of a person in a room. The indoor safety monitor hasthe same structure as the monitor of the invention of this application,and is shown, for example, in FIG. 12.

First, a lighting pattern 123 is projected from a lighting patternprojecting means 121 to a body 122 of a person to be monitored. Thelighting pattern 123 projected to the body 122 is continuously picked upas an image by an image pickup means 124, and, from two frames of imagesdifferent in time acquired by the image pickup means 124, an inter-framemoving amount of the lighting pattern 123 resulting from a movement ofthe body 122 is calculated by the moving amount calculating means.

Thereafter, in the moving amount waveform generating means, movingamount waveform data is generated by arranging the inter-frame movingamounts calculated by the moving amount calculating means in timeseries. Further, in the waveform detecting means, a body movement andrespiratory movement of the person to be monitored are detected from themoving amount waveform data.

The indoor safety monitor is provided with a safety deciding means. Fromthe body movement and respiratory movement of the person detected by thewaveform detecting means, the safety deciding means decides whether theperson to be monitored is safe or not. When the safety deciding meansdecides that the person is in a dangerous state, the informing meansautomatically informs a third party the person to be monitored is in adangerous state.

In the moving amount calculating means, an inter-frame moving amount ofa lighting pattern on an image in an axial direction connecting thelighting pattern projecting means 121 and the image pickup means 124 iscalculated from two frames different in time among images acquired bythe image pickup means 124.

In the waveform detecting means, a waveform indicating a periodicpattern is detected as a respiratory pattern, whereas a waveformindicating a variation having a high peak is detected as a body movementpattern among the moving amount waveform data generated by the movingamount waveform generating means.

The safety deciding means decides that a time zone where a body-movementpattern of the person to be monitored disappears indicates a state wherethe person stops moving, and, in this time zone, the safety decidingmeans decides that the person is in a dangerous state when the period ofthe respiratory pattern falls into disarray in a short time, when theperiod of the respiratory pattern suddenly changes, or when therespiratory pattern continuously disappears.

The indoor safety monitor mentioned above works effectively especiallywhen a monitoring region is limited. For example, this is extremelyadaptable for monitoring a person having a bath as shown in FIG. 13.Likewise, this works effectively to monitor the safety of a person in abathroom.

The present invention is not limited to the aforementioned embodiments.

It is permissible to project light that has been subjected to amplitudemodulation and extract a modulating signal from a picked-up signal.Thereby, it is possible to reduce an influence by which general light,such as indoor illumination light, is changed into a noise.

If precision is sacrificed to some degree, the frame may be a field.Therefore, the frame set forth in the appended claims has a broadconcept including a field.

INDUSTRIAL APPLICABILITY

As described in detail in the foregoing description, according to theinvention of this application, it is possible to provide a monitor byimage measurement capable of reliably detecting the respiratory movementof a sleeping person and easily making a quantitative evaluation of thedetected respiratory movement without being influenced by the posture ofthe sleeping person, illumination light, and design of a quilt.

The invention of this application is strongly expected to be put intopractical use as a technique that can support the achievement of swiftemergency treatment when a senior citizen or sick person falls into acrisis condition.

All publications, patents, and patent applications cited in the presentspecification are to be employed in the present specification asreferences without any changes.

1. A monitor comprising: lighting pattern projecting means forprojecting a specified lighting pattern; image pickup means, whoseoptical axis is different from that of the lighting pattern projectingmeans, for picking up light of a projected wavelength continuously;moving amount calculating means for calculating an inter-frame movingamount of the lighting pattern from two frames of images acquired by theimage pickup means at different times; moving amount waveform generatingmeans for generating moving amount waveform data comprising inter-framemoving amounts arranged in time series; and waveform detecting means fordetecting a movement of an object to be picked up from the moving amountwaveform data; wherein the lighting pattern projecting means projectsthe lighting pattern that has a plurality of lighting spots or slitrays; and the moving amount calculating means calculates an inter-framemoving amount of each lighting spot in an axial direction connecting thelighting pattern projecting means and the image pickup means whileproviding a positive or negative sign in accordance with a movingdirection, and calculates a sum of moving amounts of pixelscorresponding to the plurality of lighting spots or slit rays.
 2. Themonitor as set forth in claim 1, wherein the waveform detecting meansdetects a body movement and respiratory movement of a sleeping person,the monitor further comprising: safety deciding means for deciding thesafety of the sleeping person from the body movement and respiratorymovement of the sleeping person; and signal outputting means for, whenthe safety deciding means decides that the sleeping person is in adangerous state, outputting a signal indicating it.
 3. The monitor asset forth in claim 2, wherein the moving amount calculating meanscalculates an inter-frame moving amount in a partial region of thelighting pattern in an axial direction connecting the lighting patternprojecting means and the image pickup means, and determines a region inan image where an inter-frame moving amount in the partial regionappears much greater than a predetermined amount or where an inter-framemovement in the partial region appears frequently beyond a predeterminedamount as a region where the sleeping person exists, and the safetydeciding means decides that the sleeping person is in a dangerous statewhen the region where the sleeping person exists moves in a shorter timethan a predetermined time and moves frequently beyond a predeterminedamount.
 4. The monitor as set forth in claim 2, wherein the movingamount calculating means calculates an inter-frame moving amount in apartial region of the lighting pattern in an axial direction connectingthe lighting pattern projecting means and the image pickup means, anddetermines a region in an image where an inter-frame moving amount inthe partial region appears much greater than a predetermined amount orwhere an inter-frame movement in the partial region appears morefrequently than a predetermined frequency as a region where the sleepingperson exists, and the safety deciding means decides that the sleepingperson is in a dangerous state when the region where the sleeping personexists is brought closer to one end of the bed than a predetermineddistance.
 5. The monitor as set forth in claim 2, comprising informingmeans for, based on a signal output from signal outputting means,informing a third party that the sleeping person is in a dangerousstate.
 6. The monitor as set forth in claim 5, wherein the informingmeans informs the third party that the sleeping person is in a dangerousstate by voices, characters, symbols, intensity of light includinginterior illumination light, or vibrations through a telecommunicationsline.
 7. The monitor as set forth in claim 2, wherein the waveformdetecting means detects a periodic pattern as a respiratory pattern, anddetects a variation having a high peak as a body movement pattern fromthe moving amount waveform data.
 8. The monitor as set forth in claim 7,wherein the safety deciding means decides that the sleeping person is ina dangerous state when a period of the respiratory pattern falls intodisarray in a short time, when the period of the respiratory patternsuddenly changes, when the respiratory pattern continuously disappears,or when the body movement pattern frequently appears in a short time. 9.The monitor as set forth in claim 7, wherein from among respiratorypatterns, the waveform detecting means detects a value having a positiveor negative sign corresponding to a movement in a direction from thelighting pattern projecting means to the image pickup means on an imageand a value having a positive or negative sign corresponding to amovement in a direction from the image pickup means to the lightingpattern projecting means on the image as an expiratory pattern and as aninspiratory pattern, respectively, or vice versa.
 10. The monitor as setforth in claim 9, wherein the safety deciding means decides whetherrespiration belongs to normal respiration, Cheyne-Stokes respiration,central hyperventilation, ataxic respiration, or Kussmaul respirationfrom expiratory patterns and inspiratory patterns and decides that thesleeping person is in a dangerous state when the respiration belongs toany one of Cheyne-Stokes respiration, central hyperventilation, ataxicrespiration, and Kussmaul respiration.
 11. The monitor as set forth inclaim 1, wherein the waveform detecting means detects a periodic patternas a respiratory pattern, and detects a variation having a high peak asa body movement pattern form the moving amount waveform data.
 12. Themonitor as set forth in claim 11, wherein the safety deciding meansdecides that the sleeping person is in a dangerous state when a periodof the respiratory pattern falls into disarray in a short time, when theperiod of the respiratory pattern suddenly changes, when the respiratorypattern continuously disappears, or when the body movement patternfrequently appears in a short time.
 13. The monitor as set forth inclaim 11, wherein from among respiratory patterns, the waveformdetecting means detects a value having a positive or negative signcorresponding to a movement in a direction from the lighting patternprojecting means to the image pickup means on an image and a valuehaving a positive or negative sign corresponding to a movement in adirection from the image pickup means to the lighting pattern projectingmeans on the image as an expiratory pattern and as an inspiratorypattern, respectively, or vice versa.
 14. The monitor as set forth inclaim 13, wherein the waveform detecting means counts zero crosses wheresigns are reversed between an expiratory pattern and an inspiratorypattern and counts respirations.
 15. The monitor as set forth in claim14, wherein the waveform detecting means calculates a respiration numberper unit time or a respiration period from the counted respirations. 16.The monitor as set forth in claim 13, wherein the waveform detectingmeans calculates a respiration number per unit time or a respirationperiod by conducting a frequency analysis of the moving-amount waveformdata or the sampling data obtained by sampling respiratory patterns, bycalculating the most prominent frequency from a resulting frequencyspectrum, and by calculating the respiration number per unit time or therespiration period from this frequency.
 17. The monitor as set forth inclaim 1, wherein the waveform detecting means calculates a respirationnumber per unit time or a respiration period by conducting a frequencyanalysis of the moving-amount waveform data or the sampling dataobtained by sampling respiratory patterns, by calculating the mostprominent frequency from a resulting frequency spectrum, and bycalculating the respiration number per unit time or the respirationperiod from this frequency.
 18. The monitor as set forth in claim 17,wherein the waveform detecting means conducts a frequency analysis withdiscrete Fourier transform or discrete wavelet transform.
 19. Themonitor as set forth in claim 1, wherein the lighting pattern projectingmeans is disposed directly above a part close to one end of an edge ofbedding facing a head or a foot of a person sleeping on the bedding, andthe image pickup means is disposed directly above a part close to anopposite end thereof.
 20. The monitor as set forth claim 1, comprisingpresence-in-bed detecting means for detecting the presence/absence of asleeping person.
 21. The monitor as set forth in claim 1, comprising apressure sensitive switch that can be placed under the sleeping personand can be used to detect the presence/absence of the sleeping person.22. The monitor as set forth in claim 1, wherein the moving amountcalculating means calculates an inter-frame moving amount of a lightingpattern in an axial direction connecting the lighting pattern projectingmeans and the image pickup means.